QBReadR: Contextualizing NFL Throwing Decisions Through Modeling Receiver Choice

Lou Zhou

Rice University

Zachary Pipping

University of Florida

Karim Kassam

Teamworks, Outside Advisor

Motivating Example - PJ Walker to DJ Moore

Motivating Questions

  • What would other quarterbacks do in this situation?
  • Which quarterbacks deviate from the most likely option the most?
    • Do some quarterbacks find success in this deviation?
    • Should some quarterbacks throw more typical passes?
  • Look to build a ranking model to determine the most likely throw target

Data Overview

  • 2024 NFL Big Data Bowl – 2022 Season Weeks 1–9
  • Game and Play Data – Teams, Score, Play Description, Game Context, Play Result, Changes in Win Probability
  • Player Play Data – Statistics for each player for a play
    • Route ran by player, Whether the player made a tackle or interception
  • Tracking Data - Locations of players and the football at each frame of a play
  • Exclusively looking at throwing plays with an obvious target
    • Removing spikes and throwaways

Current Spacing Tells an Incomplete Story

Speed and Orientation as a Proxy for Future Separation


Methodology

  • Building a ranking algorithm(i.e. XGBoost) using hand-craft features to rank the likeliest recipient at a frame - 51.6 \(\pm\) 0.8% top-1 accuracy
    • Using a random hyperparameter search and 5-fold cross validation, with folds on matches
    • Performs significantly stronger than naive random guess(20%) and choosing the player who is farthest from their closest defender(31%)
  • Applying model to contextualize individual QB decisions by comparing them to model-predicted choices

Feature Set

Feature Category Features
Recipient Features - Distance (x, y, magnitude)
- Speed Differences (x, y, magnitude)
- Orientation Differences
- Speed Vector Differences
- Receiver Position
- First Down Indicator
Quarterback Features - Distance from Receiver
- Movement Vector
- Under Pressure Indicator
Game Context - Quarter
- Down and Distance
- Score Differential
- Time Remaining




Italics denote feature taken relative to the top-5 closest defenders

In Our Example, The Likely Play is the Safe Play



Passing the Jameis Winston Test





Potential Optimization with More Conventional Passes

Discussion

  • Able to model the likely target using an XGBoost Ranking Model with strong predictive power
    • Model can still be made stronger by incorporating newer factors like receiver skill
    • Can then contextualize throws with comparisons to the likely target
  • Comparing YPA and Completion Percentage of Throws when throwing to the likely receiver vs. other receivers is far from perfect
    • Since QBs will face different game states, some may be faced with more situations where the likely throw is the only good option
    • YPA treats interceptions and incompletions the same, potentially overvaluing players who throw risky, interception-prone throws
  • Future work should be done to model yardage and completion percentage for potential receivers to determine the optimal decision
    • Can use ranking model to determine whether it is common to find this optimal decision
    • Rewarding players who find strong but hard to see passes
  • Should also account for the relative ranking of receivers in future work, as we currently treat plays with one clear target as the same as those with multiple viable options

Further Information

Appendix A - Code

https://github.com/zachbtw/Zhou-Pipping-CMSA

Appendix B - Projecting Future Locations With a Point Estimate

Appendix C - Defender Positioning and Receiver Distance are Most Important in Deciding Target